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ABSTRACT 

The motivation for this work emerges from a desire to explain the 
experimental observation that a surprisingly small sample size vis-a-vis 
dimension is needed to achieve good signal-to-interference ratio (SIR) 
performance with an adaptive predetection filter. The adaptive filter re- 
quires estimates as obtained by a recursive stochastic algorithm of the in- 
verse of the filter input data covariance matrix. The SIR performance with 
sample size is compared for the situations where the covariance matrix esti- 
mates are of unstructured (generalized) form and of structured (finite 
Toeplitz) form; the latter case is consistent with weak stationarity of the 
input data stochastic process. It is argued that a recursive stochastic algo- 
rithm operating with a short update period(i.e. , with statistically correlated 
input data vectors) naturally constrains the covariance matrix estimates to be 
of Toeplitz form and therefore should realize any gain achievable with a (correct) 
structure assumption. The expected SIR performance for a generalized covariance 

matrix estimate is shown to approach the optimum SIR as = (1 - — 1 ^), where 

S S 

N is the filter dimension and is the sample size. For a constrained Toeplitz 

covariance matrix estimate, the expected SIR performance is shown to approach 

the optimum SIR as •: [1 - where it'(N;B) = A + BfnN + 1/2M, 

S S 

where A is Euler's constant and lsB<<N is the input data stochastic process 
correlation time. The constrained Toeplitz covariance matrix estimate tlierefore 
operates with an "effective sample size" ^15 af^d offers the 

potc'fitial of hioh expected SIR at a sample size for which tlie generalized 
estimator may provide o.cecdiivly ooor results. Insight is also provided into 


the effect of the specific fom; f>f the der.iicd signal on exnoctorl SIR performance. 


Although the work presented is cast within a specific problem framework that 
arises in sonar, radar, or seismic signal processing, the results also apply 
to the areas of pattern recognition concerned with nonparametric and para- 
metric pattern classifier parameter estimation, e.g., biomedical image recogni 
tion and earth resource satellite multispectral data classification. 
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A problem of considerable importance is that of detecting a known 
signal imbedded in an additive interference noise stochastic process. The 
physical implementation of the optimum predetection filter requires that certain 
characterizations of the additive interference noise stochastic process be a priori 
known. For example, if we constrain the filter structure to be causal and 
linear, employ an output signal-to-interference ratio (SIR) optimality cri- 
terion, and assume the interference noise covariance is a priori known, the 
optimum filter is the Wiener filter. 

With specific regard to the detection problem as it arises in the 
sonar, seismic, or radar fields, it is not possible to assume that the in- 
terference noise covariance is a priori known. A reasonable approach then 
is to estimate the completely unknown interference noise covariance required 
for the implementation. This technique leads to an adaptive filter, which 
designs itself based on estimates formed from the input data. A rather 
popular selection for the adaptive filter mechanics is offered by the stochas- 
tic version of the gradient search method [1-3]. 


For the first order, linearly constrained gradient search algorithm, it 
is well known that the weight vector (filter) iterates are asymptotically un- 
biased, provided that the step size parameter is upper bounded by the inverse 
of the maxiimim eigenvalue of the input data covariance matrix. The conver*- 
gence requirement for the variance of the weight vector iterates is consider- 
ably more demanding; it is required that the step size be upper bounded by 
(approximately) the inverse of the trace of the input data covariance matrix. 
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Therefore, the variance of the weight vector iterates and, consequently, 
the achievable filter SIR, are directly related to the filter dimension N. 
Theory establishes that in order to maintain the variance of the weight 
vector iterates at an acceptable level, we must have a small step size when 
N is large. This situation in turn suggests a possibly unbearable conver- 
gence time encompassing a large number of statistically independent input 
data vectors. The form of the optimum Wiener filter involves the inverse 

of the input data covariance matrix and appears to reinforce the belief that 

2 

a long convergence time is necessary, because it would appear that N ele- 
ments (or, at best ^(N+1) eler.ents, if symmetry is invoked) of this matrix 
must be estimated. 

Recent work [4] provides qualitative arguments indicating that the 

gradient search algorithm operating with a short update period A based solely 

on the frequency content (i.e., Nyquist criteria) of the input data stochastic 

2 

process does not pretend to attempt to estimate the N elements of the input 
data covariance matrix. The sequence of input data vectors obtained at the 
update period A is generally highly correlated; the use of these vectors 
causes the estimate of the input data covariance matrix employed by the 
gradient search algorithm to be constrained to be of (finite) Toeplitz form. 

A 

We denote the Toeplitz form estimate as Rj(*). The situation is illustrated 
in Figure 1 with specific parameters utilized to aid the visualization. Ihe 
stochastic sequence {_X(iA)) or, more compactly, {.X(i)) i=0,l,2,..., represents 
the input data sequence with the input data vector ^[(i+l)A] formed from ^(iA) 
by simply shifting the temporally oldt.^st sample out of X(iA) and shifting in 
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the most recent sample. Also shown in Figure 1 is the generalized form 
input data covariance matrix estimate Rq(*) that results when the update 
period is lengthened to La, where La is an integer multiple of the input 
data stochastic process correlation time, i.e., the stochastic sequence 
{X^(iLA)> i=0,l,2,..., is mutually uncorrelated. 

We see that operation at the short update period is implicitly con- 
sistent with the oft-invoked assumption of weak stationarity or quasi-sta- 
tionarity, i.e., weak stationarity over some interval of time, of the input 
data stochastic process and leads to a Toeplitz form input data covariance 
matrix estimate which requires only N elements for complete description. 
Operation at the short update period is generally considered in practice 
to be normal operation, with operation at the long update period of academic 
interest primarily because it leads to a much more tractable mathematical 
treatment of the statistical moment properties of the weiglit vector iterates. 
The implication of the above observation is that for a fixed real time interval 
the transient behavior of the weight vector iterates should be better for 
operation at the short update period vis-a-vis operation at the long update 
period, assuming that the input data stochastic process is indeed weakly sta- 
tionary. We do note that the reverse should be true of the steady state be- 
haviors, simply because operation at the short update period involves large 
numbers of highly correlated input data vectors. This latter point is really 
of little practical concern, i.e., the weight vector iterates ore generally 
always in the transient state for real sonar, seismic, or radar data. 


I 
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There is experimental evidence which appears to substantiate these 
qualitative arguments [4]. In the referenced v/ork» real active sonar data 
was processed utilizing a first order, linearly constrained gradient search 
algorithm. The filter dimension utilized was N=256 (complex, i.e., analytic 
signal samples) and the short update period A=. 1/5W, where W denotes the 
bandwidth of the input data stochastic process. Quite good filter SIR per- 
formance was obtained with a step size setting leading to a gradient search 
algorithm averaging time which encompassed only a small number N^eN of sta- 
tistically independent input data vectors. Results of this nature are very 
important in signal detection problems where rapid convergence demands must 
be met. The suprisingly small sample size required is conjectured to be 
a result of two factors: (1) possibility of proximity of the filter SIR 

to the optimum SIR although the weight vector iterates 
may not be near the optimum Wiener weight vector and (2) operation of the 
gradient search method with a short update period wherein the input data 
covariance matrix estimates employed are constrained to be of Toeplitz form. 

We test these conjectures in this work. The starting point is the deri- 
vation of the expected filter SIR performance for input data co- 
variance matrix estimates of both the generalized and constrained Toeplitz 
forms. The expected SIR is evaluated for several input data true covariance 
matrix examples of interest. Ihe results are conipared as a function of sample 
size for two particular values of N v/ith some generalization given for other 


values of N. 
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2. SIGNAL-TO-INTERFERENCE NOISE RATIO PERFORMANCE CRITERION 

With regard to a detection processor in which the primary goal is 
a decision as to whether a desired signal is present (M^) or absent (H^) 
in interference noise, the processor output signal -to-interference noise 
power ratio (SIR) serves as a reasonably good performance criterion. The 
interference noise consists of ever-present ambient noise plus noises at- 
tributed to other than the desired signal source, e.g., shipping traffic 
noise or reverberation noise in the passive and active sonar cases, respec 
tively. 


Using traditional binary hypothesis notation, the detection problem 
may be expressed as, 

Hq.' X(i) = N(i) 

11^: X(i) = S + N{i) (1) 

where X^(i) and N(i) are the NXl-dimensional stochastic (real) input data and 

interference noise vectors obtained at the i^^ sampling instant and S is the 

NXl-dimensional a p riori known (real) signal vector. The total signal time 

duration is Na, the extent of the time interval I . on v/hich the detection 

d 

problem is defined. The interference noise vector stocliastic process is simply 

modeled as zero-mean and weakly stationary on a time interval several times tlie 

length of and otherwise is assumed completely unknovai. We call 

the longer time interval I ; it is within r that the interference noise 

e c 

vector stochastic process must be estimated. 








(3) is seen to be equivalent to that obtained with replaced by R, the ' 

zero lag input data covariance matrix? save for a constant bias under H,. 

^n* 

We make this replacement in the ensuing calculations; this is equivalent to the 
assumptiontthat is true for iA c 1^. p i Hi^! 

To establish the relationship pertinent to this work, we let, 









where a is a scalar and R is a nonsingular, unbiased estimate of R. 

Precisely put, we have W = W(N;N^) and ^ » '^(N;Nj), but we maintain the 
dependencies on dimension and statistically independent sample size im- 
plicit, for notational convenience. Taking the expectation of Equation (^), 
we obtain, 


E[W] ^ W* = a R'^S 


the optimum Wiener v/eight vector Substituting Equation (4) in Equa- 
tion (3), 

2 


[SIR|R] = 


S^'R'^R^{’''S 


The estimate R of the input data covariance matrix may be written as, 

'R = R + E 

where E is a zero-mean stochastic error natrix. From Equation (7), vie have, 

'U'^ = [1 + R"^E]"^ R"^ 

? 

z [I - R'^E + (R'^E) ] R"^ 

taking up to the quadratic term in R'^E in the flcumariP series expansion of 
“-1 

R . Subst i luti ng Iquation (3) in Equation (6) and iiia intaininq ennsi';! enr y 


vnLli the quadratic approxirution in Fquation (3), \.-e have. 
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rsssss- 


[SIRI'^ 2 8W*^% - 3W*V* 

- 3 _Z 

W*W (9) 


utilizing the optimum weight vector of Equation (5) with a (arbitrarily) set 
equal to unity. The result of Equation (9) provides a most useful means whereby 
the expected SIR may be evaluated conditioned on various elements of the set of 
nonsingular unbiased covariance estimators Wc mention that the approxima- 

tion of Equation (8) holds well for mode*ate values of SIR and is biased slight- 
ly high at low SIR. Furtlicrmore, we deem it unnecessary to examine the condi- 
tional variance of the SIR, because tlie largest stochastic contributions will be 
to fourth order in R~^E. 
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3 . RELIEV ING THE CONDITION FOR SPECIFI C COVARIANCE MATRIX SIRDCTU RLS 

In this section, we relieve the condition of Equation (9) for two 
covariance matrix estimator structures of interest: (1) a generalized 

estimator 1lg(N^) and a constrained Toeplitz estimator 'R-j (Nj.) of R. The 
manipulations assume that the available input data sequence {^(i)) iAf.I^, 

say, i=0,l ,2, . . . ,N£-1 , is ^’'l••va^iate) Gaussian, zero-mean, mutually 
uncorrelated, and weakly stationary. The Gaussian assumption facilitates 
the evaluation of the fourth order moments that arise in the calculations. 

Case 1 : The generalized estimator assumes no particular structure -or 

the input data covariance matrix, save for syintnetry; it is given by, 

Ns-1 

1^j.(Ns) = -J- :: X(n)X^(n) N^>N (10) 

'S n=0 


It is v;ell known [5] that this estimator is the maximum 
likelihood estimator (MLE) of the input data covariance matrix R over the 
domain of positive definit*' NxH - di Miens ional matrices. Inserting Equa- 
tion (in) in Equation (9) and taking the expectation, we have. 


r[51Rlf{'^,] 


i 


— I VTtrVw c.,w /LrT. .(n)r,.. (n)] 

r. Ill i j k . ^ 


11,-1 f! 


^ f — X k rw.*w.*Wi/w ^L[r. -(n):;, (n)] 

N. "('.;* ‘i.'v;*) n 1.1 i j k 1 1 .1 ' u I 


(II) 
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n 


where w.* denotes the element o-^ W* and*^. .(n) = [x.(n)x,(n)] 


ij 


with x.(n) the i^^ element of )[(n). The scalar o.. is the (i,j) 

I • %j 

1 f 

element of R~ ; the matrix R has the (i,j) ^ element given by 
To evaluate Equation (11) we need the fourth order moment expression 
for the (weakly stationary) Gaussian stochastic variates, i.e.. 


i \th 


P-:^ P 


EC^ij.(n)^^j,^(m)] = { 


ij 


n m 
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Utilizing Equation (12), Equation (11) reduces to the very simple and 
revealing form, 


E[SIR|'^g] £ W*%* 


(lyil-n) 


(13) 


irrespective of R and 

We note that the expected SIR for a covariance matrix estimate of gen- 
eralized form is 0.8 times the optimum SIR (i.e., within = 1 dB) given 
by W*^Rlf' [cf. Equation (3) evaluated at W = W*] when 11^ = 5N+20s5N, for 
large N. The indication is that a considerably smaller sample size is 
required to achieve most acceptable expected SIR vis-a-vis v/hat would bo 
required to estimate the matrix R (or R~^) to provide a refined estimate 
of the optimum v/cight vector. Thus, the first conjecture is dispatched. 

The behavior of Equation (13) is similar to th-' result obtained by Allais as 
discussed by I'anal, et. al. [11] for the minimum mean squared error (MMSE) 
associated with th . MLE of thp ideal predictor as*'uming Gaussian statistics. 
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Case 2 : The constrained Toeplitz form estimator assumes a structure 
consistent with the weak stationarity of the input data stochastic 
process; the elements of 'R-p(N^) are given by. 


Nc N-Ij-k| 


' irrqmq-) 


S 

I E 
q=l 


Xq('')>‘qi'lj-k| j,k=l,2....N 


(14) 


We utilize the same notation for the elements of the covariance matrix esti- 
mator as in Case 1 with no fear of confusion. Inserting Equation (14) 
in Equation (9) and taking the expectation, we obtain. 


ECSIRI^t] s5U*^RW* 


J_ r 

N^2 ^ rn i j k A \ • 

N-|i-j|N-|k-i.| 

E E 

p=l q=l 




Ns"! N r 

~T '^'>:TYT w-*w *w.*w * ( /M .- n- ' i 

;-/* RW*) n m i j k Ji ^ ^ I M J 


N 2(w*'rw*) 


jr)(N-rk-tTT ’ 


W-|i-j|N-|K-z| 
E E 
p=l q=l 




( 15 ) 
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Appealing to Equation (12), we may cast Equation (15) in the fom, 


E[SIRlR^] s W*V* 

N 

1 I z Z I v/.*a-KW *[a(|i-j| ,|k-£| ) + b( | i-j j , | k-£ [ )] 
■ T 5 i j k Jl 1 J*" ^ 


. N 

z z z z w.*w.*w-*w *[a(|i-j| ,|k-A| ) i- b( | i-j | , | k-^| )] 

N5(W*'rW*) i j k £ 1 K j X 


(16) 


where 


, N-|i-j|N-|k-t| 

a(|i-j|.|k-k|> <’lp-q|'’|p-q+| 1 .jj.|l(.j|| 


06a) 

b(|i-j|.|k-0) 6 "iP-q-lk-’inp-q+li-Jll 

(16b) 


The quantities a(0» b(*) serve to account for the fact that the distinct 
elements of Rt(N<-) are obtained by summing along the diagonal and sub- 

diagonals of li'^(fl^) for each input data vector. There are no more tlian 
(H- 1 i-j 1 ) (N- 1 k-r.j )/2 distinct terms in a(*) and h(*)> because thn symmetry 

of R gives rise to the relations, 
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a(|i-j| .|k-i|) = a(ik-t| ,|i-j I ) (17a) 

b(|1-j|,|k-ii|) = b(|k-tl.|i-jl) (17b) 

It is difficult to reduce Equation (16) further; however, bounding argu- 
ments and the aid of Equation (17) permits the following to be established, 

E[SIRfRg] < ECSlRl'R^] (18) 

when the approximation of Equation (8) holds for Case 1 

It is important to note that with the op- 
timum SIR held constant. Equation (13) depends simply on N^, N; v/hereas. 
Equation (16) depends also on the Toeplitz form input data covariance matrix 
R and the signal vector S^. Tlie bound of Equation (18) is thus over all R 
and (non-trivial ) We desire a more concrete idea of how much the Case 2 
results upper bound those of Case 1 for situations of practical interest; 
the next sections assist in this regard. 
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4. SEVERAL EXAMPLES OF FINITE TOEPLIT7. FORM 

The approach we take is to select five examples of input data co- 
variance matrices of finite Toeplitz form and evaluate Equation (16) for 
each, comparing the results obtained with Equation (13). The initial results 
presented are for N-5; this dimension avoids the non-inductive results that 
sometimes obtain with two dimensional examples, but is not so large as to re- 
quire exhaustive computational time. All covariance matrix examples are 
normalized in the sense that the diagonal elements are unity. 

Example 1 ; White Process 

In this case, v/e have R=I, the NxN - dimensional identity matrix. 

This situation corresponds to an input data white noise stochastic 
process (correlation time &). We use this example for comparative 
purposes, realizing that rarely does this situation obtain, because 
the detection processor will generally be proceeded by a filter 

» 

which will increase the correlation time. I 

I 

Ex ample 2 : Tridiagonal Correlation | 

I 

I 

! 

This example provides a model for R which permits first sub-diagonal j 

correlation only, viz., 

p 

1 P o 

p 1 P 

• • • 

• » » 

« « • 

O P 1 p 

p 
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To insure that R is of positive definite form, care must be taken in 
selecting the value of p. The eigenspectrum {X) of R is readily computed 
to be, 

X^ = 1 - 2|p!cos (-^) n=l,2,...,N (20) 


From Equation (20) we see that we must have 

1 

|p| s rrr 

2 cos {^) 


( 21 ) 


in order for R to be of positive definite form; furthermore, 1p|<1/2 
if R is to be of positive definite form for all N. For this example, 
we choose the elements of R as, 

/ 


P == 

"ij " ‘’h-Jl 


0.3679 

1.0 

0 


|i-jl = 1 
|i-j| = 0 

otherwise 


i,j=l,2,...,N=5 


( 22 ) 


Example 3 : Markov Process 

The Markov process is one of extreme practical interest; the (con- 
tinuous) covariance of the input data stochastic process is of ex- 
ponential form, i.e.. 


p ( t ) = e ^ ^ ^ ^ 


Y>0 


( 23 ) 


We consider two situations, which differ in terms of stochastic process 
correlation time: 2 nd y= 5*0; the first has a correlatioti time 


reproducibiuty of the 
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twice that of the second. It can be shown [6] that the eigenspectrum {X} 
of the corresponding input data covariance matrix is given by, 

1 - p2 

^n T n=l,2,...,N 

1 - 2pcoS(|)^ + p^- 


(24a) 


where the ij)^, n=l,2,...,N; 0<4)i < 4«2 ... <(|'|^<ti, are the solutions of. 


sin[(N+l)4.|^] 

sTnT 


2psin(N4> ) P^sin[(N-l)((> ] 

+ IL =0 n-l,2,...,N 


sinifi 


n 


sinij) 


n 


(24b) 


and 


for this example, we choose the elements of R as. 


/l.O |i-j| = 0 

0.6065 |i-j| = 1 


p 


ij 





0.3679 

0.2231 


(^0.1353 


|i-jl = 2 Y = 2.5 
M-j| = 3 
|i-j| = 4 


fl-0 

0.3679 

0.1353 

0.0498 

\0.0133 


|i-j| = 0 
|i-jl = 1 
|i-j| - 2 Y = 5.0 
|i-j| = 3 
|i-j| = 4 


(24c) 


(25a) 


(25b) 
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Example Periodic Process 

When the input data stochastic process is periodic v/ith period Na, 
the resulting covariance matrix is a circulant, i.e., 


Po P] P2 


’N-1 


R = 


’ N -1 ^0 ^1 


N-2 


(26) 


Such matrices are a special type of Toeplitz matrix, arise in tlie 
modeling of certain spatial-temporal interference noise field models 
[4], and are utilized to explain the asyiaptotic (N-^+w) behavior of Toeplitz 
matrices. We may write R in the following manner [7], 

N=1 

R = L' P 
n=0 " 

where 

0 1“‘0 0 
j ^ 0 0-;‘0 0 

0 0*‘‘0 1 
_1 0-*»0 0 . 

From the eigenvectors of J, we directly obtain the (normal ized)eigeti- 
vectors E n=l,2,...,N, of R, 


(27a) 


(27b) 
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E ■= N' J 
-n 


k = 1 ,2,. . . ,N 


(28) 


The eigenspectrum {X} is given by, 


N 


^ Pk-1 ° 

n k=i ' 


■2Trj(n-1)(k-l)/N 


p 

= p„ + 2pi cos -2Ti(n-l) 

0 I 1 - 


L N 


+ 2po cos r4Ti(n-l ) 

L N . 


n=l,2,...,N 

(29) 


invoking the symmetry of R. The l3St term in the continued series of 
Equation (29) depends on whether N is even or odd. We see from Equation (29) 
that the eigenvalues of a circulant matrix are simply given by the discrete 
Fourier transform (DPT) of the first row of elements. For this example, 
we choose the elements of R as. 






1.0 

0.3679 

0.1353 




|i-j| = 0 
M-j| = 1 
|i-j| = 2 


(30) 


I 
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5. PERFORMANCE EVALUATION 

Figure 2 presents the results of evaluating Equation (13) and 
Equation (16) for the four covariance matrix examples of Section 4 as 
a function of sample size N^. In the calculations, the optimum SIR is 
fixed at unity, viz., W*^RW* = 1. The (Case 2) curves of Figure 2 are 
dependent on the choice of signal vector as mentioned in Section 3. 

The discussion below provides some insight into the detection performance 
behavior as a function of ^ and the rationale for the choice of S used 
for the curves of Figure 2. 

A 

Vie find that the quantity E[SIR[R-j.] appears to be close to its 
minimum v/hen the N elements of ^ are chosen to be equal, e.g., ^ = 1, 

A 

each element equal to unity. The quantity E[SIRlRj] is close to its 
maximum if the first element of S^ is chosen as unit with the remaining N-1 
elements zero; we denote this signal vector as S^ = A heuristic argument 
for these extrema may be given in terms of the asymptotic (fi-^+®) properties 
of the input data stochastic process. We note that the covariance matrix ex- 
amples of Section 4, with the exception of Example 1, correspond to lowpass 
input data stochastic processes, which have continuous power spectral density 
(psd) functions. A periodogram estimate of the psd (under the zero mean, 
Gaussian assumption) is not consistent; speci fical ly, the asymptotic variance 
at a frequency is proportional to the square of the psd at that frequency [8]. 
A choice S = 1^ accentuates the zero frequency component of the input data 
stochastic process, for which the spectral estimation variance is greatest; 


thus, the detection performance with such a signal Is expected to be poor. 

A selection of S ■ Sl_ causes all frequency components of the lowpass stochas- 
tic processes to be utilized, thereby reducing the statistical variability 

A 

of E[SIR|Rj] through an averaging. This situation Is strongly exhibited 
in the case of Example 1, and less so for Example 2, wherein a choice S « $1 
gives rise to good detection performance by permitting the constrained Toeplltz 
form estimator of Equation (14) to utilize the statistical Independence of- the 
elements wi thin each Input data vector X(n) to further reduce estimate variance. 
Such a choice Is therefore expected to provide nearly optimum detection per- 
formance for finite N and no signal bandwidth constraint. If the eigenvalue 
dispersion (condition number) of the input data covariance matrix Is small. 

The situation Is not unlike the use of diversity to Improve communications 
system performance. 

In view of the above, the signal vector admits a useful representation 
In terms of the members of the complete orthonormal (COH) set of (real) eigen- 
. vectors associated with the input data covariance matrix R, I.e., 

I 

j • 

j where the expansion coefficients are given by, 


(31b) 
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When s^“l n=l,2,...,N, ^possesses unit length along each principal 
component associatou with the input data stochastic process. For finite 
N, this choice for ^ parallels the asymptotic arguments that obtain when 
S=S1 , except for the case of Example 1, for v/hich £ degenerates to 1_, 
causing poor detection performance. The exclusion of Example 1 is ex- 
plained simply by noting that the white stochastic process may be expanded 
in terms of any^ CON set. The poor detection performance v;e refer to obtains 
when E^ has unity in the n^*^ position and zeroes elsewhere; however, ue can 
clioose the CON set (E) differently so that the v/hite stochastic process, and 
as in Equation (31a) with the expansion coefficients equal to unity, does 
not give rise to anomalous detection performance. For S as in Equation (31a) 
with the expan«ilnn coefficients equal to unity, the optimum v/cight vector 
is calculated from Equation (5) (with a=l) to bo, 

W* = E X-' (32) 

n=l ' 

where {X} is the eigenspectrum of R. Since ^ shov/s no preference for one 
principal component vis-a-vis another, the optimum v/eighting for a principal 
component is simply inversely proportional to the respective eigenvalue. 

The Case 2 solid curves of Figure 2 are for ^ as computed from Equa- 
tion (3l) with the expansion coefficients equal to unity. In arriving at ^ for 
the covariance matrix examples sho\/n, the eigenvector set {£} was computed 
for each of tlie examples. The eigenspectrum {X) v.'as also computed for each of 
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the examples and checked against Equations (20), (24), and (29). The 

TnI 

eigenvalues are arranged In descending order (over |^| the case of 
Example 4) Inverted, scaled to a maximum of unity, and displayed In Figure 3 
as the optimum v/elghting applied to each principal component. The Case 2 
dashed curves of Figure 2 are for Examples 1 and 2 when S=S1 . 


The data presented In Figure 2 may bo coalesced In a most meaningful 


form. Examination of Equation (13) Indicates that the expected SIR for the 


"s-N-'l 


T 


the trend in the 


generalized covariance matrix estimator varies as| 
curves of Figure 2 suggests that the expected SIR for the constrained Toeplltz 
covariance matrix estimator varies witli samirlr. size In the following manner, 


E[SIRfR^] ^W*^RW* 


N' -N-4 \ 


(33a) 


where the "effective sample size" N' Is given by, 

N' = P>1 (33b) 

and P=P(S), i.e., p carries implicit dependence on S. Evaluating the 
tiiultipl icdtive factor p from Equation (33) and the data of Figure 2, we 
obtain the results shown in Table 1. 
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Covariance 

Matrix 

Example 

P 

3 (Mp, Y=rn)) 

3 (MP, y=2.5) 

4 (PP) 

1.06 

2.38 

2.41 

1.87 

2.53 


TABLE } 

EFFECTIVE SAflPLE SIZE INCREASE 
OF TOEPLITZ ESTIMATOR OVER 
GENERALIZED ESTIliATOR, N=5 


We infer from the values of Table 1 that, on ine average, a sample siz. re- 
diictioM of 2.12 is possible with the constrained Toeplitz estimator vis-a-vis 
the generalized estimator for N=S. This result is highly significant when the 
input data stochastic process is quasi-stationary, because the (real) time 
necessary to achieve good detection performance is reduced to approximately 
0.47 times that normally required. 

Since the above results are tendered on a specific dimensionality, we 
make an attempt to establish the dependence of B on N. lo do tliis, v/e repeat 
the calculations for N=10. The larger Example 1 and 2 r;atriccr. arc natural 
extensions of those in Section 4; the elements oF the larger Example 3 mt- 
rices are readily computed from Equation (25) with A=0.1, v«=2.5 and 5.C; 
and the Example 4 matrix elements are given by. 
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Pij = P i-j 




'l.O 

|i-j| 

0.6065 

M-j| 

0.3679 

|i-j| 

] 0.2231 

|i-j| 

0.1353 

M-Jl 

^0.0821 

|i-jl 


= 0 
= 1 
= 2 
= 3 
= 4 
= 5 


(34) 


Figure 4 displays the expected SIR curves for N=10, and Figure 5 
illustrates the optimum principal component weighting. Generally, we 
observe the same trends in Figure 4 as exhibited by the curves of Figure 2, 
except for the results pertaining to the Example 2 (tridiagonal) covariance. 

In this case, the difference in detection performance obtained with S as in 
Equation (31) and S=$l is nrticeably less at the higher dimension. The in- 
ference .s that asymptotic arguments appear to hold for the Example 2 co- 
variance at the moderate dimension N=10, i.e., the eigenvectors of the Ex- 
ample 2 covariance matrix are apparently near those of Equation (28). EvjIp 
ating the multiplicative factor 3 from Equation (33) and the data of Figure 4, 
v/e obtain the results shown in Table 2. 
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Covariance 

Matrix 

Example 

B 

1 (WP) 

1.7Z 

3.86 

1 (WP- S=S1 

2 CTDT 

2 (TO, S=S1) 

IM 

3 (MP, y=5.0) 

3.17 

3 (MP, y=2.5) 

2.16 

4 (PP) 

3.57 


TABLE 2 

EFFECTIVE SAI-1PLK SIZE INCREASE 
OF TOEPLITZ ESTIMATOR OVER 
GENERALIZED ESTIMATOR, N=10 


We infer from the values of TaDle 2 that, on the average, a sample size re- 
duction of 3.12 is possible with the constrained Toeplitz estimator for N=10. 

We now attempt to generalize detection performance behavior for 

Case 2 and N>10; in doing this, we obtain a rather interesting result. The 

expected SIR of Equation (16) may be solved in closed form when the input data 

stochastic process is white (Example 1) and S=S1 . Performing the calculations, 

/■ 

we obtain, 


:lSIR|RjJ 


H W* RW* 
WP,S1 


^(M)'^s ~ ' ^00 

^(N)^'^s 


(35a) 


where 


N 

if(N) = 


n=l 


]_ 

n 


( Suit ) 


= A ^ f.ii N + 1/2N 
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and A = 0.577 is Euler's constant. The harmonic series ti(N) exhibits a 
behavior for small N and is divergent with an asymptotic approach consistent with 
the natural logarithm. The result of Equation (35) is felt to hold for a wide 
variety of signal vectors v/hose psd's are relatively flat over the unconstrained 
bandwidth. Examination of the behavior of Equation (16) appears to indicate' that 
the approximation to the series tt(H) is modified to n'(N) in the following manner 
when the input data stochastic process is not white, but the input data vectors 
still contain a large number of degrees of freedom, 

ti"(N;B) = a + BanN + 1/2N B^l (36) 

where Ba is the input data stochastic process correlation time, e.g., B=2 
for the Example 2 (triadiagonal ) case. Now, comparing Equation (33) to Equa- 
tions (35) and (36) we obtain a generalization for the multiplicative factor 

B. i.e., 

' 8 ~ 

^ = ti'(N;B) 

N 


A + B«.nN H 1/2N 


( 37 ) 



Several observations concerning Equation (37) are worth mentioning: 

(1) use of Equation (37) in Equation (33) with B selected appropriately provides 
results which compare favorably with the data of Figures 2 and 4 and Tables 1 and 
2, and (2) the multiplicative factor B is divergent. The second point is very 
significant, indicating that a sample size < N for which the generalized es- 
timator'^Q of Equation (10) is not even of full rank may be equivalent to an ef- 
fective sample size which provides good detection performance when used with 
the constrained Toeplitz estimator of Equation (14). An example will aid in 
understanding these remarks and also serve to dispatch the second conjecture put 
forth in Section 1. For the short update experimental results mentioned in Section 
1, the filter dimension N=256. The correlation time of the input data stochastic 
process is approximately Ba = 8/5W, where W denotes the bandwidth of the stochastic 
process; thus, there are approximately 32 uncorrelated time segments within each 
input data vector. The value of B as computed from Equation (37) is 5.70. A 
sample size = N = 256 then provides us with an expected SIR of .827 times the 
optimum SIR (performance equivalent to 0.82 dB from the optimum SIR) from Equation 
(33). The real time required to gather the input data necessary to achieve this 
performance is 4 s, the gradient search algorithm averaging time utilized. We 

’S 

note that for = N = 256, Equation (13) for the generalized estimator is v;ay out 

of the range for v/hich the approximation of Equation (8) is valid. ^ 
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6. CONCLUDING REMARKS 

We have examined two conjectures put forth in Section 1 pertaining 
to the suprisingly small sample size required to achieve good SIR perform- 
ance with an adaptive signal detection filter. The first conjecture involves 
the possibility of convergence of the SIR v/ell before convergence of the 
filter impulse response; the second, with the possibility that the mechanics 
of a certain adaptive filter algorithm are intelligent enough to capitalize 
on the structure of the input data stochastic process. Specifically, the 
structure referred to is the (finite) Toeplitz form of the input data co- 
variance matrix. 

The principal results of this work are as follows: 

(1) Equation (9), the expected SIR, conditioned on the estimator'^ 
of the input data covariance matrix R; 

(2) Equations (13) and (16), the expected SIR (under the stated assump- 
tions) with the condition relieved for generalized and con- 
strained Toeplitz estimators, respectively; and 

(3) Equations (33), (35), (36) and (37), synergistically providing a 
model of the behavior of the constrained Toeplitz estimator with 
dimension, sample size, true covariance matrix, and signal vector. 
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Use of the generalized estimator results In the expected SIR approach- 
ing the optimum SIR as ■ (1 - ^ where N Is the filter dimension and Nj 

s s 

Is the sample size. In this case, S' 5N + 20 samples are needed to come within 
1 dB of the optimum SIR; this Is a relatively small number of samples, being 
considerably less than that required to estimate the (N + N)/2 unique elements 
of a generalized covariance matrix. Use of the constrained Toeplltz estimator 
results In the expected SIR approaching the optimum SIR as ff'El - ] 

where ir'(N;B) behaves as BtnN, 1<B«N, for large N. This result Is derived from a 
combination of the data analyses presented In Section 5 for small -to-moderate. 

N using the covariance matrix examples of Section 4, and heuristic asymptotic 
arguments. The Importance of the results Is evident. The constrained Toeplltz 
estimator operates with an "effective sample size" NJ s^d therefore 

has the potential to provide high expected SIR at a sample size for which 
the generalized estimator may provide exceedingly poor results. The reduction 
In required sample size reflects as a reduction In the amount of (real) time 
required for the adaptive algorithm performance to converge and, therefore, 
corresponds to an increase in the ability of the filter to accomnodate realistic 
situations wherein the Input data stochastic process Is quasl-statlonary. 


The Improved performance Is naturally achieved with an adaptive algorltNn, 
e.g., a first order, linearly constrained gradient search algorithm, operating 
at a so-called slow update rate, because the covariance matrix estimate formed 
Is of constrained Toeplltz form. This observation suggests that the proposed 
scheme of Farden [9] for first forming a constrained Toeplltz form estimate of 


the Input data covariance matrix and then Inserting the estimate In the adaptive 
algorithm is unnecessary. If a slow update rate Is employed. V.'e do note that 


It i slight difference In the estimate utilized by a gradient search 
^1g pr1|ito and that of Equation (14) employed for the calculations In this work 
Mn that the (first order) gradient search algorithm Introduces exponential 
temporal weighting, whereas Equation (14) employs uniform temporal weighting. 

This difference Is second order to the calculations presented here. 

In many problems of Interest* we deal with an adaptive filter which 
operates concurrently on a number* say K, of N>d1mens1ona1 input data vectors 
)^(1) k**l, 2,...»K, at the 1^^ sampling Instant. The Index on k m^y be a 

spatial reference, I.e., time records of length Na from K sensor locations are 
processed by the algorithm. If the (spatial) sequence of Input data vectors 
are organized as In reference [4] and the resulting KN-dlmonslonal Input data 
vector stochastic process Is assumed to be weakly stationary, then the Input 
data covariance matrix If of (finite) block Toeplltz form. Extension of the 
results presented In this work to this case are Immediate. 

To broaden the areas of study that may benefit from this work, we mention 
the general area of pattern recognition. The problem discussed here corresponds 
to the situation where the form for the discriminant function Is known and the 
samples are used to estimate the values of parameters of the classifier, I.e., 
the nonparametric problem. The results provided are also useful for relating 
sample size and dimensionality for the parametric problem, I.e., the fonn of the 
underlying probability distribution Is known and the saniples are used to estimate 
the values of parameters of the classifier [10]. 
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